Forensic video recording with presence detection

ABSTRACT

At a high level, embodiments of the invention relate to augmenting video data with presence data derived from one or more proximity tags. More specifically, embodiments of the invention generate forensically authenticated recordings linking video imagery to the presence of specific objects in or near the recording. One embodiment of the invention includes video recording system comprising a camera, a wireless proximity tag reader, a storage memory and control circuitry operable to receive image data from the camera receive a proximity tag identifier identifying a proximity tag from the proximity tag reader, and store an encoded frame containing the image data and the proximity tag identity in the storage memory.

RELATED APPLICATIONS

This patent application is a continuation and claims priority benefit,with regard to all common subject matter, of commonly assigned U.S.patent application Ser. No. 17/213,919, filed Mar. 26, 2021, now U.S.Pat. No. 11,532,334, issued Dec. 20, 2022, entitled FORENSIC VIDEORECORDING WITH PRESENCE DETECTION; which is a continuation of U.S.patent application Ser. No. 14/880,453, filed Oct. 12, 2015, entitled“FORENSIC VIDEO RECORDING WITH PRESENCE DETECTION,” which is now U.S.Pat. No. 10,964,351, issued Mar. 30, 2021; which is a continuation ofU.S. patent application Ser. No. 14/517,368, filed Oct. 17, 2014,entitled “FORENSIC VIDEO RECORDING WITH PRESENCE DETECTION,” which isnow U.S. Pat. No. 9,159,371, issued Oct. 13, 2015, (“the '371 Patent”).The '371 Patent is a continuation-in-part application and claimspriority benefit, with regard to all common subject matter, of commonlyassigned U.S. patent application Ser. No. 13/967,151, filed Aug. 14,2013, and entitled “COMPUTER PROGRAM, METHOD, AND SYSTEM FOR MANAGINGMULTIPLE DATA RECORDING DEVICES” (“the '151 Application”). Theabove-referenced patent and application are hereby incorporated byreference in their entirety into the present application. Embodimentsand/or features of the invention described in the present document maybe used with the subject matter disclosed in commonly assigned U.S. Pat.No. 8,781,292, filed Sep. 27, 2013, issued Jul. 15, 2014, and entitled“COMPUTER PROGRAM, METHOD, AND SYSTEM FOR MANAGING MULTIPLE DATARECORDING DEVICES” (“the '292 Patent”), which is a continuationapplication of the '151 Application. The '292 Patent is herebyincorporated by reference in its entirety into the present application.

Embodiments and/or features of the invention described in the presentdocument may be used with the subject matter disclosed in commonlyassigned U.S. patent application Ser. No. 14/040,329, filed Sep. 27,2013, and entitled “PORTABLE VIDEO AND IMAGING SYSTEM” (“the '329Application”); and commonly assigned U.S. patent application Ser. No.14/040,006, filed Sep. 27, 2013, and entitled “MOBILE VIDEO AND IMAGINGSYSTEM” (“the '006 Application”). The '329 Application and the '006Application are hereby incorporated by reference in their entirety intothe present application.

Further, embodiments and/or features of the invention described in thepresent document may be used with the subject matter disclosed incommonly assigned U.S. patent application Ser. No. 14/517,226, filedOct. 17, 2014, and entitled “DUAL LENS CAMERA UNIT,” and with commonlyassigned U.S. patent application Ser. No. 14/517,160, filed Oct. 17,2014, and entitled “BREATH ANALYZER, SYSTEM, AND COMPUTER PROGRAM FORAUTHENTICATING, PRESERVING, AND PRESENTING BREATH ANALYSIS DATA.” Eachof these patent applications is also a continuation-in-part of the '151Application. These patent applications are hereby incorporated byreference in their entirety into the present application.

BACKGROUND 1. Field

Embodiments of the invention generally relate to augmenting video datawith presence data derived from one or more proximity tags. Morespecifically, embodiments of the invention generate forensicallyauthenticated recordings linking video imagery to the presence ofspecific objects in or near the recording.

2. Related Art

Video recordings of law enforcement activities are becoming more commonand more frequently used in legal proceedings. However, it is presentlya weakness of such systems that the recordings they produce cannot beverifiably traced back to a specific recording device or authenticatedas unaltered. Furthermore, specific objects appearing in the scenecannot be identified. This can be particularly significant when, forexample, calibration or identification records for a device need to beproduced.

SUMMARY

Embodiments of the invention address the above problems by augmentingvideo data with presence data derived from one or more proximity tags. Afirst embodiment of the invention includes a video recording systemcomprising a camera, a wireless proximity tag reader, a storage memory,and control circuitry operable to receive image data from the camera,receive a proximity tag identifier identifying a proximity tag from theproximity tag reader, and store an encoded frame containing the imagedata and the proximity tag identity in the storage memory.

In a second embodiment of the invention, a method of recordingauthenticated video with presence data comprises the steps of creatingan augmented encoded frame by encoding video data into an encoded frame,receiving one or more proximity tag identifiers from a proximity tagreader, including the received proximity tag identifiers as metadata forthe encoded frame to produce the augmented encoded frame, generating adigital signature for the augmented encoded frame, and storing theaugmented encoded frame and digital signature.

In a third embodiment of the invention, a computer-readable mediumstoring an augmented video file is disclosed, the video file comprisinga plurality of augmented frames, each augmented frame of the pluralityof augmented frames including video data and one or more identifierseach associated with a proximity tag.

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the detaileddescription. This summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter. Other aspectsand advantages of the current invention will be apparent from thefollowing detailed description of the embodiments and the accompanyingdrawing figures.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

Embodiments of the invention are described in detail below withreference to the attached drawing figures, wherein:

FIG. 1 depicts an exemplary hardware platform of certain embodiments ofthe invention;

FIG. 2 depicts a system diagram illustrating the components of oneembodiment of the invention;

FIG. 3 depicts a flowchart illustrating the operation of one embodimentof the invention; and

FIGS. 4A and 4B depict illustrative video files produced in accordancewith one embodiment of the present invention.

The drawing figures do not limit the invention to the specificembodiments disclosed and described herein. The drawings are notnecessarily to scale, emphasis instead being placed upon clearlyillustrating the principles of the invention.

DETAILED DESCRIPTION

The subject matter of embodiments of the invention is described indetail below to meet statutory requirements; however, the descriptionitself is not intended to limit the scope of claims. Rather, the claimedsubject matter might be embodied in other ways to include differentsteps or combinations of steps similar to the ones described in thisdocument, in conjunction with other present or future technologies.Minor variations from the description below will be obvious to oneskilled in the art, and are intended to be captured within the scope ofthe claimed invention. Terms should not be interpreted as implying anyparticular ordering of various steps described unless the order ofindividual steps is explicitly described.

The following detailed description of embodiments of the inventionreferences the accompanying drawings that illustrate specificembodiments in which the invention can be practiced. The embodiments areintended to describe aspects of the invention in sufficient detail toenable those skilled in the art to practice the invention. Otherembodiments can be utilized and changes can be made without departingfrom the scope of the invention. The following detailed description is,therefore, not to be taken in a limiting sense. The scope of embodimentsof the invention is defined only by the appended claims, along with thefull scope of equivalents to which such claims are entitled.

In this description, references to “one embodiment,” “an embodiment,” or“embodiments” mean that the feature or features being referred to areincluded in at least one embodiment of the technology. Separatereference to “one embodiment” “an embodiment”, or “embodiments” in thisdescription do not necessarily refer to the same embodiment and are alsonot mutually exclusive unless so stated and/or except as will be readilyapparent to those skilled in the art from the description. For example,a feature, structure, or act described in one embodiment may also beincluded in other embodiments, but is not necessarily included. Thus,the technology can include a variety of combinations and/or integrationsof the embodiments described herein.

Embodiments of the invention may be embodied as, among other things amethod, system, or set of instructions embodied on one or morecomputer-readable media. Computer-readable media include both volatileand nonvolatile media, removable and nonremovable media, and contemplatemedia readable by a database. For example, computer-readable mediainclude (but are not limited to) RAM, ROM, EEPROM, flash memory or othermemory technology, CD-ROM, digital versatile discs (DVD), holographicmedia or other optical disc storage, magnetic cassettes, magnetic tape,magnetic disk storage, and other magnetic storage devices. Thesetechnologies can store data temporarily or permanently. However, unlessexplicitly specified otherwise, the term “computer-readable media”should not be construed to include physical, but transitory, forms ofsignal transmission such as radio broadcasts, electrical signals througha wire, or light pulses through a fiber-optic cable. Examples of storedinformation include computer-useable instructions, data structures,program modules, and other data representations.

Different forms of computer-readable media store data in different ways.For example, volatile storage media such as RAM may retain data only aslong as it is powered, while non-volatile media such as flash memoryretain data even when powered off. Furthermore, some forms of computerstorage media are write-once, read many (WORM), such that data can bestored to them but not erased or overwritten. For some forms of WORMmedia, data can be recorded in multiple sessions, where the data fromone session is appended to the data from the previous session. Otherforms of media may be indefinitely rewriteable. Some forms of media maybe encrypted, such that data is written to them encrypted by anencryption key (which can correspond to the device, the user, or beunique in some other way) and data read from them is scrambled unlessdecrypted with the corresponding decryption key.

Additionally, storage media can be made tamper-resistant such that it isdifficult or impossible to alter or erase data stored to them, or toprevent reading data except by authorized means. WORM media or encryptedmedia, as described above are one way to make storage media tamperresistant. Another way is to make storage media physically difficult toremove, such as by covering them with epoxy after they have beeninstalled. Other methods of making storage resistant tamper resistantare also known in the art and can be used.

A first broad class of embodiments of the invention includes a videorecording system comprising a camera, a wireless proximity tag reader, astorage memory, and control circuitry operable to receive image datafrom the camera, receive a proximity tag identifier identifying aproximity tag from the proximity tag reader, and store an encoded framecontaining the image data and the proximity tag identity in the storagememory.

Turning first to FIG. 1 , an exemplary hardware platform that can serveas, for example, the control circuitry or other elements of certainembodiments of the invention is depicted. Computer 102 can be a desktopcomputer, a laptop computer, a server computer, a mobile device such asa smartphone or tablet, or any other form factor of general- orspecial-purpose computing device. Depicted with computer 102 are severalcomponents, for illustrative purposes. In some embodiments, certaincomponents may be arranged differently or absent. Additional componentsmay also be present. Included in computer 102 is system bus 104, wherebyother components of computer 102 can communicate with each other. Incertain embodiments, there may be multiple busses or components maycommunicate with each other directly. Connected to system bus 104 iscentral processing unit (CPU) 106. Also attached to system bus 104 areone or more random-access memory (RAM) modules.

Also attached to system bus 104 is graphics card 110. In someembodiments, graphics card 104 may not be a physically separate card,but rather may be integrated into the motherboard or the CPU 106. Insome embodiments, graphics card 110 has a separate graphics-processingunit (GPU) 112, which can be used for graphics processing or for generalpurpose computing (GPGPU). In some embodiments, GPU 112 may be used forencoding, decoding, transcoding, or compositing video. Also on graphicscard 110 is GPU memory 114. Connected (directly or indirectly) tographics card 110 is display 116 for user interaction. In someembodiments no display is present, while in others it is integrated intocomputer 102. Similarly, peripherals such as keyboard 118 and mouse 120are connected to system bus 104. Like display 116, these peripherals maybe integrated into computer 102 or absent. Also connected to system bus104 is local storage 122, which may be any form of computer-readablemedia, and may be internally installed in computer 102 or externally andremoveably attached.

Finally, network interface card (NIC) 124 is also attached to system bus104 and allows computer 102 to communicate over a network such asnetwork 126. NIC 124 can be any form of network interface known in theart, such as Ethernet, ATM, fiber, or Wi-Fi (i.e., the IEEE 802.11family of standards). NIC 124 connects computer 102 to local network126, which may also include one or more other computers, such ascomputer 128, and network storage, such as data store 130. Local network126 is in turn connected to Internet 132, which connects many networkssuch as local network 126, remote network 134 or directly attachedcomputers such as computer 136. In some embodiments, computer 102 canitself be directly connected to Internet 132.

Turning now to FIG. 2 , a system diagram illustrating the components ofone embodiment of the invention is depicted; the depicted system isgenerally referred to by reference numeral 200. In this embodiment, onecomponent is camera 202, which captures imagery of a scene including oneor more objects. In some embodiments, camera 202 is a still camera. Inother embodiments, camera 202 is a video camera. In still otherembodiments, a plurality of cameras may be simultaneously capturingimagery of the scene. Camera 202 may be a visible-spectrum camera, aninfrared camera, a millimeter-wave camera, a low-light camera, or anyother form of imaging device now known in the art or hereafterdeveloped. Camera 202 may also include a microphone to capture audioalong with video data.

In this example, the objects in the scene include a suspect 204, apolice officer 206, and a patrol cruiser 208. Some objects in the scenemay be tagged with proximity tags. Here, officer 206 is wearing a badgethat includes proximity tag 210, and patrol cruiser 208 may have anintegrated proximity tag 212. In another embodiment, a proximity tagassociated with a user is embedded in a credit card-sized proximity cardthat can be carried in the user's wallet. Furthermore, objects notvisible in the scene may also have proximity tags. For example, a secondofficer standing out of view may have a proximity tag, or officer 206may have a service weapon that is not visible in the scene but has aproximity tag. Other objects in the scene, such as suspect 204, may notbe tagged, or may start untagged and later become tagged. In thisexemplary scenario, suspect 204 could become tagged by being restrainedwith handcuffs containing a proximity tag.

Any object that may need to be specifically identified if it appears inrecorded image data can be tagged with a proximity tag. Examples oftaggable objects in a law enforcement scenario include badges, serviceweapons, canine units, patrol cruisers, forensic kits, breath analyzers,radar and lidar guns, and evidence bags. It will be apparent to a personof skill in the art that the objects to be tagged will be differentdepending on the scenario.

A proximity tag such as proximity tag 210 and 212 is any device thatradiates an identifying signal, herein referred to as the proximity tagidentifier, that can be read by a corresponding reader such as proximitytag reader 214. Proximity tags can be active (meaning that theyperiodically broadcast their identifier), assisted passive (meaning thatthey broadcast their identifier only when interrogated by a signal fromthe reader), or passive (meaning that they have no power source and mustbe illuminated by a signal from the proximity tag reader in order toradiate their identifier). Other forms of proximity tags are alsopossible. Proximity tag identifiers may be preprogrammed into proximitytags, or may be field-programmable, such that the user assigns theidentifier when the proximity tag is deployed. One common form ofproximity tag system is the radio-frequency identification (RFID) tagand the corresponding RFID reader. Another form of proximity tag systemutilizes a challenge-response protocol to avoid the spoofing of aproximity tag identifier. Any form of proximity tag, now known orhereafter developed, can be used.

Proximity tag reader 214 receives the proximity tag identifierstransmitted by proximity tags such as proximity tags 210 and 212.Depending on the type of proximity tag, a different type of reader maybe required to receive the proximity tag identifiers. For example, anactive reader is required to read passive tags. In some embodiments,proximity tag reader can determine the distance to the transmitting tagbased on signal strength or other information. In some embodiments,multiple proximity tag readers are present. In some suchimplementations, positional information about the tag can be determinedbased on relative signal strength at each reader.

Also present in system 200 is one or more sensors 216. Sensors 216collect or receive data to supplement the audiovisual data provided bycamera 202. Examples of such sensors include additional microphones forrecording supplementary audio data, additional clocks for providing timedata, a radio receiver for recording radio transmissions, aglobal-positioning system (GPS) receiver for recording position data, abreath analyzer for detecting intoxication, a fingerprint reader forlogging individual identity, and one or more accelerometers forrecording movement and acceleration data. Additional sensors, such as aholster event sensor for detecting when a holster cover is opened orwhen a weapon is removed from the holster, may be directly or wirelesslyconnected.

In this embodiment, raw image data from camera 202 is first processed byencoder 218. Raw image data may be encoded by any still image or videocodec now known in the art or developed in the future. In particular,many image and video file container formats provide for the addition ofmetadata to the image or video data. Where such provision is not made,metadata can be stored in an auxiliary file and optionally linked tospecific video frames or still images using timestamp data, including atime when the image data was acquired, a filename for where theauxiliary data is stored, or similar.

Combiner 220 combines encoded video data from encoder 218 and proximitytag identity data received from proximity tag reader 220, resulting inan augmented encoded frame. In some embodiments, the identities ofmultiple proximity tags are added to a single augmented encoded frame.In some embodiments, data from sensors 216 is also added to theaugmented encoded frame. In certain embodiments, not every frame isaugmented. For example, when encoding MPEG data, identity tag data mayonly be included with I-frames. In other embodiments, identity tag datamay be included in any frame where the set of detected tags changes. Insome embodiments, data relating to the signal strength associated witheach tag may also be included with the tag identity. In some embodimentswith multiple proximity tag readers, the identity of the readerdetecting the tag and/or the received signal strength at that reader isalso included in the augmented encoded frame.

In some embodiments, augmented encoded frames are further digitallysigned by the recording device to verify that they have not beentampered with. To do this, some embodiments use a device-specific keystored in memory 222. In some such embodiments, memory 222 is tamperresistant, such that it is difficult or impossible to extract thedevice-specific key. The digital signature can be generated by signer224 using any algorithm for producing digital signatures now known inthe art or later developed, including public-key schemes such as theDigital Signature Algorithm (DSA) and keyed hash schemes such as theHash-based Message Authentication Code (HMAC).

In some embodiments, individual images or frames will be signed. Inother embodiments, entire video files are signed. In still otherembodiments, groups of frames are collectively signed. In yet otherembodiments, only those frames that contain identity tag data aresigned. In some embodiments, the digital signatures are stored asmetadata with the encoded frame. In other embodiments, the digitalsignatures are stored as metadata with the container file. In stillother embodiments, an auxiliary file containing a detached signature canbe generated, potentially with timestamps corresponding to the framesthat are signed.

In this embodiment, augmented frames, together with non-augmented frameswhen they are present, are combined into a container file format bycombiner 226 to produce a final video file. In cases where the image orvideo file format used makes no provision for embedded metadata, themultiple files containing image data, proximity tag identity data,sensor data and signature data may all be combined into a single file bycombiner 226. In other embodiments, combiner 226 may be integrated intoencoder 226. In still other embodiments, combiner 226 may re-encode,transcode, or compress data for efficient storage.

In some embodiments, a display such as display 234 may also be presentin system 200 for viewing the final video data in real time. In somesuch embodiments, display 234 may be configured to overlay data such assensor data and/or proximity tag identity data onto the video data. Inother embodiments, display 234 takes the form of a heads-up display thatoverlays sensor data and/or proximity tag data on the scene as viewed bythe user.

The video file produced (in this embodiment) by combiner 226 is storedin data store 228 for subsequent retrieval. In some embodiments, datastore 228 takes the form of tamper-proof storage so that the video data,once stored, cannot be deleted or altered. In some embodiments, datastore 228 is a remote data store to which data can be uploaded at theend of a shift or in real time. In some such embodiments, a computersuch as computer 230 may be able to access the video file in real timefor display on display 232 to a remote viewer such as a dispatcher.

In another broad class of embodiments of the invention, a method ofrecording authenticated video with presence data is disclosed,comprising the steps of creating an augmented encoded frame by encodingvideo data into an encoded frame, receiving one or more proximity tagidentifiers from a proximity tag reader, including the receivedproximity tag identifiers as metadata for the encoded frame to producethe augmented encoded frame, generating a digital signature for theaugmented encoded frame, and storing the augmented encoded frame anddigital signature.

Turning now to FIG. 3 , a flowchart illustrating the operation of oneembodiment of the present invention is depicted. The method of creatingaugmented video may be initiated manually, or by a local or remotesensor-related triggering event. Such a sensor-related triggering eventmay be generated directly by the sensor, or by a recording devicemanager, such as a Digital Ally® VuLink®, that controls and synchronizesvarious recording devices. For example, the recording device manager maycommunicate (via wireless communication, wired communication, or both)to sensors such as described herein, one or more person-mounted cameraunits, a vehicle-mounted video camera oriented to observe eventsexternal to the vehicle, a vehicle-mounted video camera oriented toobserve events internal to the vehicle, and/or one or more storageelements. In some embodiments, the recording device manager detects whenone video camera begins recording, and then instructs all otherassociated devices to begin recording. The recording device manager mayalso send information indicative of a time stamp to the variousrecording devices for corroborating the recorded data.

For example, the recording device manager may instruct all associatedvideo cameras to begin recording upon the receipt of a signal from asensor such as a breath analyzer that a breath analysis has begun. Thisensures that multiple video cameras record the breath analysis, forfuture authentication that the breath analysis was performed correctly.The recording device manager may also send a time stamp to all theassociated video cameras to provide a corroboration of the variousrecorded data.

In another example scenario, an officer wearing a badge with an embeddedproximity tag enters a patrol car. Detection of this proximity tag bythe recording device manager serves as a triggering event and causescameras in the patrol car and on the officer's body to begin recording.Proximity tag identifiers associated with both the officer and thepatrol car are stored as metadata with both recordings. In this way, notonly are the officer and the patrol car associated with each videorecording, but the common proximity tag identifier data allows the twovideo recordings to be associated with each other as well.

The method begins at step 302, when image data is received. In someembodiments, the image data is a single frame of raw video data receivedfrom a video camera. In other embodiments, it is a pre-encoded frame orframes from a video camera. In still other embodiments, it is an imagereceived from a still camera. In yet other embodiments, it is image datain any combination of the forms above from a plurality of cameras.

Processing then proceeds to step 304, where the image data received isencoded into a frame. Any image-coding algorithm now known in the art orhereafter developed can be employed for this encoding step. This processcan also involve decoding, transcoding, or recoding the image data.Furthermore, if the image data was received in encoded form, noadditional encoding may be necessary at this step. In embodiments whereimage data is received from multiple cameras, this step may also involvesynchronizing, merging, and/or combining the image data from themultiple cameras. In some embodiments, image data from multiple camerasmay be combined into a single augmented frame; in other embodiments,image data from multiple video cameras is stored in separate video datafiles. In still other embodiments, features of video codecs andcontainer files allowing multiple camera angles of a single scene may beused to store separate frames containing video data from each of thecameras in the same video data file.

Next, at step 306, proximity tag identity data for one or more proximitytags is received from a proximity reader. In some embodiments, signalstrength information or other information relating to the position ofthe tag relative to the reader is also received. In some embodiments,data is received from a plurality of tag readers. In such embodiments,different tag readers may read tag identity data for some or all of thesame proximity tags, or a tag may be read by only one reader of theplurality of tag readers.

At this point processing proceeds to step 308, where supplemental datafrom a sensor such as sensor 216 is received. In some embodiments, datais received from more than one such sensor. Depending on the type ofsensor, data may be received every frame, for a subset of frames, or foronly a single frame. Different sensors may provide data at differentrates and different times, so that successive frames may have differentsets of supplemental data available.

Processing then proceeds to decision 310 where it is determined whetherto augment the encoded frame. This determination may be made based on avariety of factors. In some embodiments, every frame is augmented. Inother embodiments, only certain types of frame (such as I-frames in MPEGvideo encoding) are augmented. In still other embodiments, the framewill be augmented if a triggering signal is generated based on datacontained in the frame, the proximity tag identity data, and/or thesensor data. For example, the detection of particular tags, orparticular numbers or combinations of tags, may always trigger frameaugmentation. In general, a triggering event can be any event suggestingthat additional context about the frame should be stored.

For example, the user triggering the siren and/or light bar of a patrolcruiser might be a triggering event. Alternately, a velocity oracceleration reading, either from the cruiser or from integratedvelocity and/or acceleration sensors may be a triggering event.Similarly, a vehicle crash, detected by an accelerometer reading, airbagdeployment, or similar stimulus, might be a trigger event. Additionally,a positional reading could be a triggering event. Such a positionalreading could be absolute (for example, entering or exiting a particulargeofenced area) or relative (for example, moving more than a particulardistance from a patrol cruiser or other fixed or mobile point ofreference). A further example of a triggering event is an indicationfrom a sensor configured to detect when a holster cover is opened orwhen a weapon is removed from the holster. Another form of user-relatedtriggering event could come in the form of one or more biometric stressindications (such as elevated heart rate, blood pressure respiration,etc.) obtained from biometric sensors worn by the user. Similarly, audiodata could generate triggering events if raised voices or high levels ofvocal stress are detected.

Triggering events can also come from the context of the data beingcollected. For example, when encoder 218 detects that the video data itis processing contains a face, a triggering event could be generated.Alternately, this functionality could be limited to the recognition of aparticular face (for example, if the user sees a face matching aphotograph provided with a warrant, or on a wanted poster, a triggerevent could be generated). Similar recognition algorithms can be appliedto other data streams as well; for example, the audio signature of agunshot could be a triggering event, or the positional signature ofevasive maneuvering.

Additionally, triggering signal can be generated manually by the useror, in embodiments where data is streamed to a remote date store, by aremote observer. Of course, a person of skill in the art will recognizethat a wide variety of triggering signals are possible and thatvariations and combinations of the above will become apparent.

If the determination is made to augment the frame, processing proceedsto step 312; otherwise, processing continues at decision 314. At step312, some or all of the proximity tag data and supplementary sensor datais stored as metadata for the encoded frame, creating an augmentedencoded frame. As discussed above, some video codecs have provision forencoding frame-level metadata. In such embodiments, sensor data andproximity information can be stored directly with the encoded frame. Inother cases, it may be desirable to store the proximity tag data andsupplementary sensor data as metadata for the video file as a whole. Inthis case, it may be advantageous to store a timestamp or other frameidentifier with the data to associate different sets of supplementaldata with their respective encoded frames. In other embodiments, it maybe advantageous to leave the video file unaltered and store theproximity tag data and supplementary sensor data in a separate metadatafile. Again, storing timestamps or frame identifiers associated with thedata may be useful.

Processing now proceeds to decision 314. At this point, a determinationof whether to authenticate the (augmented or non-augmented) frame ismade. This decision can be based on the same factors as thedetermination of whether to augment a frame; however, if thecomputational cost of calculating a digital signature is high, it maynot be feasible to calculate a digital authentication for every frame.In some embodiments, only some of the frames are digitally signed. Inother embodiments, sets of frames are digitally signed as a group. Instill other embodiments, an entire video file is signed as a singleunit. If the determination is made to generate a digital signature,processing proceeds to step 316.

At step 316, the digital signature is generated. As discussed above, anydigital signature or authentication code algorithm can be used toproduce these digital signatures. Next, at step 318, the digitalsignature is stored. As with the metadata discussed above, digitalsignatures can be stored with the corresponding frame, with thecorresponding video file, or in a separate file. In some embodiments,the digital signature is only calculated for the encoded frame; in otherembodiments, the digital signature is calculated for the encoded frametogether with any metadata relating to it.

After the digital signature is stored, or if the determination was madeat decision 314 not to generate a digital signature, the frame is storedin the container video file at step 320. Any container file format canbe used. For example, if the encoded frames are encoded using the MPEG4Part 10 codec, which is commonly referred to as H.264 codec, a varietyof container files including 3GP, DivX, MP4, and MPEG Program Stream canbe used. Different video codecs can also use these container fileformats or a variety of others. If the original image data was stillimage data, or if auxiliary files are used for storing metadata ordetached digital signatures, an archive file format such as ZIP may beused. In some embodiments, nested container file formats can be used,such as a ZIP file containing an MP4 file containing video, an XML filecontaining proximity tag and sensor data, and a PGP signature filecontaining a detached signature.

Finally, at step 322, the container file is written to memory (such asdata store 228). In some embodiments, the container file is not writtenout until video recording completes. In other embodiments, the videofile is progressively written out as additional frames are generated. Instill other embodiments, intermediate container files are generated andstored, and a final “closed” container is written out once recordingterminates. Once the container file is written out, processing returnsto step 302, where additional image data can be processed.

In another broad class of embodiments of the invention, acomputer-readable medium storing an augmented video file is disclosed,the video file comprising a plurality of augmented frames, eachaugmented frame of the plurality of augmented frames including videodata and one or more identifiers each associated with a proximity tag.

Turning now to FIGS. 4A and 4B, two embodiments of an augmented videofile are depicted. In FIG. 4A, an augmented video file 402 that storesproximity tag data, sensor data, and digital signature data as metadatafor the associated frame is depicted. Frame 404 is an augmented frame,including metadata fields for proximity tag data 406, sensor data 408,and digital signature 410, which authenticates proximity tag data 406,sensor data 408, and image 412. Also included in frame 404 is the actualimage data 412.

Also present in file 402 is frame 414, which is an augmented butnon-authenticated frame. Accordingly, there are metadata fieldsincluding proximity tag data 416 and sensor data 418 in addition toimage data 420. The last depicted frame in file 402 is frame 422, whichis a non-augmented, non-authenticated frame. Accordingly, only imagedata 424 is present (in addition to whatever metadata fields areordinarily generated in the process of encoding).

FIG. 4B, by contrast, depicts an augmented video file 426 where allmetadata is stored in auxiliary files. This may be desirable where, forexample, original, unaltered video files are needed for evidentiarypurposes. In this embodiment, augmented video file 426 is a ZIP filecontaining video data file 428, which is in MP4 format, proximity tagand sensor data file 430, which is in XML format, and digital signaturefile 432, containing ASCII-encoded PGP signatures. A person of skill inthe art will immediately recognize that other arrangements and fileformats are possible; for example, proximity tag data and sensor datacould be in separate files.

Video data file 428 contains image data frames 434, 436, and 438. Insome embodiments, these could be identical to image data received fromthe camera. In other embodiments, these could be re-encoded ortranscoded. In this embodiment, proximity tag and sensor data file 430contains proximity tag and sensor data in XML format. Present with eachset of proximity tag and sensor data is a frame identifier associatingan image data frame such as image data frame 434 or 438 with thecorresponding proximity tag and sensor data.

Finally, video data file 428 contains digital signature file 432containing detached signatures for one or more video frames and/or thecorresponding proximity tag and sensor data. Again, a frame identifiercan be included with each digital signature to associate it with thecorresponding image data frame.

Video data file 428, alone or in combination with other augmented ornon-augmented video files can then be used to document a scene ofinterest. In some cases, post-processing of one or more video data fileswill be employed to better reconstruct such a scene. In a lawenforcement scenario, for example, at the end of a shift, a large volumeof video data may be collected from officers returning to the stationhouse. If an incident of interest occurred during that shift, then itmay be desirable to immediately collect all relevant recordings. In somecases, this can be done by searching the metadata embedded in each videofor one or more proximity tags relevant to the incident (such as, forexample, a proximity tag associated with an officer, vehicle, or objectknown to have been present at the scene). In this way, all of theaugmented video data pertaining to the incident can easily be collectedwithout error or significant labor.

Similarly, augmented video data can easily be searched for any videodata matching certain criteria. For example, video data recorded in acertain area can easily be located by searching any location data (suchas GPS metadata) stored in or with the video data for correspondingcoordinates. Recordings of DUI stops can easily be obtained by searchingfor video data where a proximity tag associated with a breath analyzeris present. All video recordings of an officer can be quickly collectedby searching video data for a proximity tag associated with thatofficer's badge. Many other applications will quickly become apparent toa person of skill in the art.

Although embodiments of the invention have been discussed in a lawenforcement context, a person of skill in the art will realize thatnumerous other applications immediately present themselves, includinginventory and stock control, gaming, and surveillance. Accordingly, theinvention is not limited to the context discussed, but can be applied ina wide variety of scenarios.

Many different arrangements of the various components depicted, as wellas components not shown, are possible without departing from the scopeof the claims below. Embodiments of the invention have been describedwith the intent to be illustrative rather than restrictive. Alternativeembodiments will become apparent to readers of this disclosure after andbecause of reading it. Alternative means of implementing theaforementioned can be completed without departing from the scope of theclaims below. Certain features and subcombinations are of utility andmay be employed without reference to other features and subcombinationsand are contemplated within the scope of the claims. Although theinvention has been described with reference to the embodimentsillustrated in the attached drawing figures, it is noted thatequivalents may be employed and substitutions made herein withoutdeparting from the scope of the invention as recited in the claims.

Having thus described various embodiments of the invention, what isclaimed as new and desired to be protected by Letters Patent includesthe following:
 1. A method of associating image data with proximity, themethod comprising: receiving the image data from one or more image datarecording devices; receiving proximity tag data associated with aproximity tag from a proximity tag reader, wherein the proximity tagdata comprises signal strength information indicative of a position ofthe proximity tag relative to the proximity tag reader; augmenting aframe of the image data to include a first portion of the signalstrength information corresponding to a first time of the frame; andaugmenting an additional frame of the image data to include a secondportion of the signal strength information corresponding to a secondtime of the additional frame.
 2. The method of claim 1, wherein theproximity tag is associated with a law enforcement officer.
 3. Themethod of claim 1, wherein the proximity tag is associated with avehicle.
 4. The method of claim 1, wherein the proximity tag data isassociated with a service weapon.
 5. The method of claim 1, furthercomprising: receiving an indication of a triggering event from at leastone sensor, wherein the frame of the image data is augmented responsiveto the triggering event.
 6. The method of claim 1, further comprising:parsing metadata associated with the image data for the proximity tagdata; and verifying a presence of the proximity tag within a particularframe of the image data based on the proximity tag data.
 7. The methodof claim 1, further comprising: generating a digital signature for theframe of the image data; and storing the digital signature as metadatafor the frame of the image data.
 8. A system comprising: one or moreimage data recording devices; a proximity tag reader; a proximity tag;and one or more non-transitory computer-readable media storing acomputer-executable instructions that, when executed by at least oneprocessor, perform a method of associating image data with proximity,the method comprising: receiving the image data from the one or moreimage data recording devices; receiving proximity tag data associatedwith the proximity tag from the proximity tag reader, wherein theproximity tag data comprises signal strength information indicative of aposition of the proximity tag relative to the proximity tag reader;augmenting a frame of the image data to include a first portion of thesignal strength information corresponding to a first time of the frame;and augmenting an additional frame of the image data to include a secondportion of the signal strength information corresponding to a secondtime of the additional frame.
 9. The system of claim 8, wherein theproximity tag is associated with a law enforcement officer.
 10. Thesystem of claim 8, wherein the proximity tag is associated with avehicle.
 11. The system of claim 8, wherein the proximity tag data isassociated with a service weapon.
 12. The system of claim 8, wherein theproximity tag reader is an RFID reader.
 13. The system of claim 8,wherein the one or more image data recording devices comprises aplurality of cameras.
 14. The system of claim 8, further comprising: oneor more audio data recording devices configured to record audio dataassociated with the image data.
 15. One or more non-transitorycomputer-readable media storing computer-executable instructions that,when executed by at least one processor, perform a method of associatingimage data with proximity, the method comprising: receiving the imagedata from one or more image data recording devices; receiving anindication of a triggering event from at least one sensor; receivingproximity tag data associated with a proximity tag from a proximity tagreader, wherein the proximity tag data comprises signal strengthinformation indicative of a position of the proximity tag relative tothe proximity tag reader; responsive to the triggering event, augmentinga frame of the image data to include a first portion of the signalstrength information corresponding to a first time of the frame; andaugmenting an additional frame of the image data to include a secondportion of the signal strength information corresponding to a secondtime of the additional frame.
 16. The one or more non-transitorycomputer-readable media of claim 15, the method further comprising:generating a digital signature associated with the image data.
 17. Theone or more non-transitory computer-readable media of claim 16, whereinthe digital signature is associated with the one or more image datarecording devices.
 18. The one or more non-transitory computer-readablemedia of claim 16, the method further comprising: storing a video filecomprising the image data including the frame, the additional frame, andthe digital signature.
 19. The one or more non-transitorycomputer-readable media of claim 18, wherein the video file is stored ina tamper-resistant format such that the image data is prevented frombeing edited or erased.
 20. The one or more non-transitorycomputer-readable media of claim 15, wherein the frame of the image datais augmented responsive to the triggering event.